white paper from et hernet u b iquity t o et hernet conv erge nce : the emergence of the converged netw ork interface controller (c-nic) the fo cus of th is pap e r is on th e eme r gen ce o f the converged network in terfa c e contr o lle r ( c -nic) p r o v idin g a c celerat ed client/se r ve r , clusterin g , and stor age networkin g and ena b ling the u s e o f un ifie d tcp/ip e t hernet co mmunications. the breadth an d im portan ce o f serve r app lica t io ns tha t ca n bene f it from c-n i c capabilities, togeth e r with the e m ergen ce of server o peratin g syste m s interfa c es enabling hi ghly integ r ated ne tw ork accele r a tion capa bilit ies, pr omise t o make c- n i cs a st and a r d f e a t ur e of v o lu me server configurations. c-nic de ploymen t will pro v ide d r amatic ally imp r ove d application performa n ce, scal a bil i ty and s e rve r co st-o f-o w nershi p. the unified ethernet network architecture enabled by c-nic w ill be n o n-disruptive to existing n e tworking and serve r infra s tru c t u re, while providing significant l y bett e r p e r f orman c e at redu ced co st vi s- -vis a l ternatives. march 2 0 04
it is widely recognize d that the single grea test fa ctor impacting data cente r application performa n ce an d scalability is the server network i/o bottlene ck. the fact that n e twork ba ndwidth a n d tra ffi c l o ads fo r c l i e nt/se r ve r , clus terin g and stora ge tra ffic ha ve o u tpace d an d will continue to con s istently outpace cpu perfo rman ce in cre a ses results in a significant and growing mismatch of capabilitie s that ha s b e com e one of the data cen t er m a nager?s la rgest problems . a common so lution to this challenge ha s b een to use different n e tworking techno logies o p timized for spe c ific server traffic type s, fo r e x a m ple, eth e rnet for clien t /server commu nications a n d file-b ased storag e , fibre cha nnel for block-based stora ge, a n d sp ecial purpose lo w-laten c y proto c ols for server cluste ring. however, such an approach has acqu isition and operat ional cost di ffi cult ie s, ma y be disru p ti ve to e x isting applicatio ns, and in hibits m i gr ation to n e wer server syste m to polog i es, such as blade servers and vi rtualized server sy stems. enter uni f ied ether n et com m unica tions an em erging approa ch is focu sed on evol ving u b iqu i tous gigabit ethernet tcp/ip networkin g to addre ss the req u iremen ts of client/se r ver, clu s tering, and stora ge commun i cations thr o ugh dep loyment o f a unified eth e rnet communication s fabric . the vi si on of such ne twork ar ch itec ture is that i t i s n o n-di srupt i ve t o e x i s ti ng data center in frastru c ture and provid es sign ific ant l y b e t te r p e r f o r m anc e at a f r a c t i o n of the cost ? all th e while pre s erving the ex isting inve st ment in se rver and n e twork infra s tru c t u re. up gr ading of d a ta center s wi ll be e n abled in a n evo l uti o nary fash i o n on a rack-by-ra ck ba sis using e x isting ma nagement tools and interfa c es. additionally, the approach prom ise s no mo difications to e x isting applicat io ns. at the ro ot of the em ergence o f unified e t hernet data cen t er communica tions is th e c o mi ng toget her o f thr ee net w o r k i ng t e chno l o gy trends : tcp offload engine (toe ) , r e m o te di rec t m e mo ry acc e s s (r d m a) over tcp , and i s cs i. toe refers to the tc p/i p proto c ol sta c k b e ing o f floa ded to a d edica ted controller in order to re duce tc p/i p proce s sing overhe ad i n s e rv er s eq ui p ped wi t h s t and a r d gigabit network interface controllers (nic s) . while to e techn o lo gy ha s b e e n the focu s o f si gnifi c ant v e ndor engi neering in vest ment, a numbe r of o b st acl e s re main for broad - based toe deploym e nt. the first generation of toe products has su ffered from tw o re lated d r aw backs: lack of standa r d inte rfa c e s wi thin se rver ope r a t ing sy ste m s for to e integ r ati o n and an all-in clu s ive appr oach to arch ite c ting tcp offl oad as a para llel server netw ork proto c ol stack. the recent eme r gence o f microsoft ?s chimne y offload ar chitecture for enablin g toe inte gration within win d o w s server operating system s w i ll spur the developme nt of toe products that are tig h tly cou p le d with se rver op eratin g system s fo r optimum server performan ce gains and lower cost of own e rsh i p. an add i tio n al disad v antage of existing t o e pro d ucts, which typically a r e an embedd ed co mponen t of toe n e twork interfa c e cards (t -nics), is the expense. each one o f the s e t-nics is ne eded in ea ch server, and two are need ed for hig h -
availa bil i ty con f igur at ions. cost s ar e like ly to be h i gh , appro x im ately $700 to $1 00 0 for a t-nic. rdma is a techn o log y tha t allo ws the net w ork inter f ace cont roller ( n ic), under th e control of the app lica t ion, to pla c e da ta di rec t ly i n to and out of appl ic ation memory , remo ving the need fo r data cop y ing and e n abling su pport fo r low-laten c y commun i c a tions, su c h as clu s te ring an d st orage co m m unicat i on s. rdma networkin g has until n o w con s iste d of specialized interconnects which repre s ent an exp e nsive and (fo r existing da ta cente r s) disruptive a l terna t ive due to their prop rietary and monolithic architectu re s. in e s sence, th ese requ ire a spe c ia l rdma-ena bled nic w i th a pro p rietary in terface to a h i gh-speed, prop rieta r y fa bric. for organi zation s w i t h a la rge i n vest ment in standa r d se rve r s and gig a bi t network i n g and var i ous com m e rcia l and cust om so f t ware a ppli c ation s , o r for organizatio n s with tig h t budg ets, this is an unattra c tive o p tion becau se it require s abandonin g existing infrastructu re rath er than buildin g upon it. a rang e o f networkin g indu stry leaders, in cludin g br oadco m , c i sco, de ll, emc, hp, ibm, micro s oft, and n e tapp, ha s co me tog e ther to su pport the developme nt of an rdma o v er tcp proto c ol stan dard and pro v ides fa cilities imme diately usefu l to existing an d futu re g i gabit tcp/ip-ba s ed cluste ring, stora ge, a n d othe r a pplication proto c ols. rdm a a p i s including soc k et s d i rect pr oto c ol (s dp) and wi ns ock di rec t (ws d ) giv e the ba se o f e x isting t c p/i p sockets applic a t ions ac ce s s to the be nefit s o f r d ma over tcp, while rd m a-nati v e a p is, wh ich allow appl i c ation s to take maxi mum advantag e of rd ma over tcp, have a l so been d e ve loped. the s e include the direct acce ss p r o g ramming library (d apl) de veloped by th e dat collaborative and incorporated into th e stan dardization efforts o f the i n terconn e ct stan dard consortiu m (isc) o f the open group, a s well a s the emerging window s server named bu ffers ap i. iscsi is d e signe d to enable end - to-end block st orag e netwo r king over tc p/i p gigabit networks. it is a tran sp ort p r oto c o l fo r sc si that o pera t es on top of tc p through en capsu l atio n of scsi comm ands in a tcp d a ta strea m . isc s i is emerging as an alte r n ative to p a ralle l sc s i or fib r e channel wi thin the da ta center a s a blo c k i/o t r ansp ort for a ra nge o f ap p licat ion s in cludin g s a n /nas con s oli d ati o n, messaging , databa se , and high -perfo rman ce compu t ing. application servers h a ve two a v ailable op tions for su pporting iscsi in itiators: a softw a re- o nly initi a to r, (su c h a s mi cro s o f t? s i s csi so f t ware d r i v er) wi th a standar d gigabit eth e rnet n i c or an iscsi host bu s adapter (h ba). isc s i hbas typ i cally u s e dedicated controlle rs to fully offload iscsi and tcp/i p proto c ols fro m server ho st cpu for op timum performan ce and le vera ge storag e services, such as m u lti-pa th i/o p r ovid ed b y th e server ope r ating system.
unleashin g data ce nter appli c ation pe rforma nce throug h ip prot ocol suite offload the sp ecific pe rfo r ma nce b e nefits provi ded by tcp/ip, rdma, a n d isc s i o fflo a d depend up on two bro a d app lication-specific factors: the nature of ap plication network i/o and lo ca tion of its perfo rman ce b o ttlen ecks. am o n g netwo r king i/o characteristics, avera ge tran sa ction size a n d t h r o ug hp u t a n d l a te n c y s e ns it i v it y p l ay an imp o rta n t ro le in d e term ining the value toe and r d ma brin g to a spe c i f i c application . the deg ree to which an ap pl ication is storage netw ork-bound is a crit ica l de t e rminant i n the bene f it that is c s i block storage n e twor kin g brings to it. margalla c o mmunications an alysis of workloads runni ng on ?scale-out? 1p/2p serv ers sh ows t h at i p pr oto c ol suite of flo a d bene f its span the range o f ap plica t ion s running w i thin today?s data cen t ers. as su mmarized in figu re 1 , ip prot ocol suite offl oad is e s pec iall y b e nefi cial f o r fr ont-en d web serve r app lica t io ns, back-e nd high- perfo rman ce compu t ing and decision su pp ort a pplica t ions, and common i n ternet file service (cifs) file storage service s . t h ese a pplication s taken tog e the r co mprise a majo r p o rtion of th e overall server ma rket, both in term s o f shipments and revenue, making i p proto c ol su ite o f fload the indu stry?s first re al and co m p lete solut i on to the ser v er networ k i / o pr ob le m that can be intro d u c ed in a se amless and non-disruptive manner. serv e r applica t ion tcp/ip o ffload rd ma over tcp o ffload iscsi o ffload comme nts common int e rnet f ile se r v ic e s (cifs) persi s tent, high-bandwi d t h cli e nt to fil e server connecti o ns ? back-end is csi s t o r a g e e n a b le s fi le s e r v e r c o n s o lid a t io n . web s e rvi c es toe/rdma benefi t scal e s wi th web server capaci ty. high- performance computi n g (hpc) rdma impro v es hpc clust e r scal a bil i t y ? bi osc i ences and earth sci ences appl i c ati o ns are network i/o bound. deci si o n sup p ort se rvice s deci si o n support cl i e nt-to - server and server-to-sto r age commu ni cati ons i s typi call y network i/o bound ? rdma i m pacts server cl uster scal a bili ty. fig. 1: i p protoco l suite offloa d valu e b y server w o rkload source: v o lume serv er wor k loa d analysis, marga lla communications, 2003 the brea dth and imp o rtance o f serve r ap p licat ion s th at can ben efit fr om i p suite offload, to gether w i th the emergence of server operating s y stems interf ac es enabling highly in tegrated o f floa d ca pabiliti es, promise to ma ke ip p r oto c ol suite offl oad a standard fe ature of vo lume ser v e r con f igur a t ions. th is promi s e ca n only be fu lfilled through th e eme r gen c e o f a ne w con v erg ed netw ork in terface controller (c-nic) product ca te gory wh ich goes be yo nd existing mono lith i c offloa d ap proach es
to provide unifie d to e, rdma, and isc s i offload an d dra m atically imp r o v ed appli c ati o n per form a n ce, scala b ilit y, and serve r cost -of-o w nership. converge d networ k interf ace controller (c-nic) ? the critical enabler for unified et hernet co mmunica tions the main focus of em erging c-n i c pro d ucts will be o n integ r ating ea ch o f the main hardware and softwa re compon ents of ip pr oto c ol su ite o f fload into a unified whole. specifi c al l y , an integ r ated c - ni c device w ill com p ri se the fo llo wing speci f ic capabi lit ies: ? tcp pr oto c ol o f f l oad t i ghtl y coup led w i th se rver ope r a t ing sy ste m s u s ing interf ace s such as mi cro s oft chi m ney ? rdm a o v er tcp of floa d sup p ort e d vi a r d m a int e rfa c e s su ch as soc k et s dire ct prot ocol (s dp) or wins ock d i rect ( w sd) ? iscsi offlo a d which is tightly co upled w i th server ope r ating system stora g e stac k. ? embedde d server ma nagement capability with suppo rt for os p r esent and absent st a t es ? enabl i ng comp le te re mote sy stem s m a nagemen t fo r ra ck, blade, an d virtualized server systems twi sted p a i r net w o rking tra ff i c management traffic storage t raffic c l uste r i ng tra ff i c s erver with c -ni c s erver w or k stat i on s torag e lan management cluster ipc storage convergence over ethernet fig. 2: c - n i cs con c ur rently a c celerat e eth e rnet tc p/ ip-ba s ed n e tworking, cluste ring, and stora ge tra ffic. the requ irement for seamless integratio n of th e mu lt iple fa cet s of c- nic functionality w i th server opera t ing syst em environme n ts is a critical produ c t require m e n t; ?ou t-o f-the-bo x? a cce leration of existing and new a pplication s usin g standard t c p/i p and scsi interfaces will b e a p r ere q u isite for c - nic de plo y ment.
a single in tegrate d c - nic de vice with associated so ftware w ill support a c celerated perfo rman ce for gen eral data n e tworking, clu s tering , and stora ge commu nication. as shown in figu re 2, c-nic-e q u ipped se rvers will h a ve the ca pability to concu rrently suppo rt the three different tra ffi c type s acro ss one or mult ipl e se rve r gigabit po rts, tru l y en abling unified end - to-end gigabit ethern et-based commun i c a tions. c-nic value proposition summary c-nics will allow dat a cent er a d ministrat o r s to ma xi mize the value o f a v a ilable server resource s. i t will allow server s to share gbe network p o rts fo r a ll type s o f traffic, re move netw ork ove r he ad, si mpli fy e x isting network ca bling, an d faci lita t e infusion of server an d network technology upgrad es. the s e b e n efits a r e provided with no ch anges to a pplication softw a re a n d while m a intainin g the e x istin g use r and softw a re managem ent interfa c es. in sum m ary, c-nic s provide th e fo llow i ng imp o rtant bene f its: ? increased server and network performance. co m p ar ed t o ex i s t i ng gigabit ni cs, c-nic s allow fu ll overhe ad of ne twork i/o pro c e ssin g to be remo ved from the server. in add i tion, a ggregation o f networkin g , sto r age, and clustering i/o o f fload in to th e c- nic fu nction re m o ves netw ork ove r he ad and s i gni f i c antl y in cr ease s e ffe cti v e ne tw ork i/o, w h ile impr o v ing co st- effe ctivene ss versus alternative discre te o fflo a d a pproache s . ? simplified server additions and d a t a center ne twork u p grades. si nc e network i/o fun c tion s ha ve b e e n a ggre g ated, newly added c - n i c-equip pe d servers, su ch a s b l ad e se rve r s, only need to be conn ected to a single giga bit ethernet connection. con v erse ly, upg r ading to 10 g b e only re quires the addition o f a m o dule to the server. ? improved operational efficiency. by us i n g c- ni cs , t h e i n t e rf ac es to eac h server an d rack a r e simplifie d . there are fewer conn ection po in ts, fewer cable s , fe wer ada p ter cards, a n d ea sier upgrade s to existing networks. changes a r e lo calize d to the c-nic. fewe r chan ges translate to im pro v e d efficien cy. in ad ditio n , for so m e data ce n t ers, more produ c tive se rve r s can result in fe wer server s, which re duce acqu isition and ongoing m a intenan c e and mana gement expense s.
br oadcom?, the p u l se l ogo, connecti ng ev er y t hi ng?, and the connecti ng ev er y t hi ng l ogo ar e tr adema r ks of br o a dcom cor por ati on and/ or i t s aff i liat es in the united st ates and c e rta i n o t her countr i es. a l l other tr adema r ks or tr adenames mentioned ar e the pr op er t y of thei r respe c ti v e ow n e r s . bro a d c o m c o rp or a t ion 16215 alton park w a y, p. o. bo x 5 7013 irvine, california 92619-70 13 ? 20 04 by br oad c o m c o r p or at i o n . al l r i g h ts r e ser v ed. n g st - w p 1 00- r 09 .03 . 200 4 phone: 949- 450- 8700 fax: 9 49-45 0-87 10 e-mail: info@bro adcom.com web: www .b road com.com
|
